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Ph ■ 1. Introduction 

*£h" ■ 

Let (f2, J 7 , F = (J r s)t<s<oo, P) be a filtered probability space satisfying the usual conditions and 

SW be an M. d valued Brownian motion adapted to F. Consider the following stochastic differential 
, ■ equation in R n 

dX s = b(s,X s ,a s )ds + a(s, X s ,a s )dW s , (1.1) 

\Q ■ 

where at the control belongs to A, the set of all progressively measurable processes with values 
CN ! in a compact subset A of R fc . 

Let O C R n be a bounded open set, and set Q = [0,T) x O. For a given initial (t,x) € Q, 

define r as the first exit time of the M n+1 -valued process (s, X s ) from the bounded domain Q, 
^ | that is 

T = mf{s>t:(s,X s )(^Q}. (1.2) 

Given a running cost function ^ : M + x M n x A — > 1R and a terminal cost function g : M + x M n — > R, 
we define the value function as 

j^i F(t,x) = M^y* e(s,X a ,a a )ds + g(T,X(T))} , (1.3) 

in which is the expectation operator conditional on = x. Occasionally, we will refer to X 
as X t,x to emphasize its initial condition. 

In general one can show that the value function is a viscosity solution of a fully non-linear 
Hamilton- Jacobi-Bellman equation given that it is a continuous function; see Corollary 3.1 on 
page 209 of [1]. However, when the domain is bounded, it is not always the case that the value 
function is continuous due to tangency problem mentioned in |11| pp. 278-279], which imposes 
continuity as an additional assumption. Consider two underlying processes X 1 = X f ' x (solid line) 
and X 2 = X^ x (dotted line) in Fi gure Q3 No matter how close X 1 and X 2 are, the difference 
between their first exit time t\ and ti could be very large. 
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Figure 1. Tangency problem 

A sufficient condition for the continuity of the value function is provided on page 205 of [3]. 
In this paper we improve this condition using a probabilistic argument; see Theorem 14.11 and 
Example 14,11 We also note that the regularity of the stochastic exit time control problem has 
been studied in [12J, in which the value function is shown to be Lipschitz continuous assuming 
the existence of an appropriate "global barrier" . Under weaker assumptions, similar to the ones 
considered here, the continuity of the value function was obtained by [T] and [5] for semi-linear 
and quasi-linear Dirichlet problems, respectively, using purely PDE methods. More recently, the 
continuity of viscosity solutions of fully non-linear Dirichlet problems (with integro-differential 
terms) is analyzed in [2j. Related results can also be found in [8], where the Dirichlet problem for 
the Isaacs Equation is discussed. With respect to these aforementioned papers our contribution is 
to give a simple probabilistic proof of the continuity result for the fully nonlinear Cauchy problems 
on bounded domains. 

The rest of the paper is organized as follows: In Section [2] we recall some preliminary results. 
Section EJ is devoted to an important result on the sample path behavior of the state process on 
the boundary of the domain of the problem. Using the results developed in Section [3l a sufficient 
condition on the continuity of the value function is derived in Section |H Some of the proofs are 
given in the Appendix. 

2. Preliminaries 

This section presents definitions and assumptions needed for the setup of our problem, and 
collects some relevant classical results. 

To proceed, we present the standing assumptions needed for our work. Below we use | • | for 
the absolute value of a scalar and | ■ | for the second Euclidean norm. 

Assumption 2.1. For any x,^ 1 ,^ 2 E M n , a E A, t E [0, T], functions b,a,£, and g satisfy, for 
some strictly positive constant K 

(1) \\b(t,x l ,a) - b(t,x 2 ,a)\\ + \\a(t, x 1 , a) - a(t, x 2 , a)\\ < K \\x x - x 2 ||; 

(2) ||6(*,a?,o)|| + \\a(t,x,a)\\ < K(l + ||x||), V(t,x,a) E [0,T] x R n x A; 

(3) I and g are continuous functions; 

(4) \£(t,x 1 ,a)-£(t,x 2 ,a)\ + \g(t,x 1 )-g(t,x 2 )\ < K\\x l - x 2 \\; x\x 2 E R n ,{t,a) E [0,T] x A; 

(5) \£(t,x,a)\ + \g(t,x)\ < K (1 + \\x\\ 2 ) . 

The first two of our assumptions guarantee that (jl.ip has a unique strong solution for a given 
a £ A. 

Next, we present the dynamic programming principle; see e.g. [H 113]. 



Proposition 2.1. For any stopping time 6 with t < < t, 

V(t,x)= ME tiX U e(s,X s ,a s )ds + V(9,X e )\ . (2.1) 

Let Vv9 G C rl ' 2 (Q) 

G a <p(t,x) = ip t {t,x)+L?tp(x), 

and 

= 6(i, x, a) • D x (p(t, x) + - tr (<7cr'(i, s, a).D^(t, x)). (2.2) 

Using the dynamic programming principle it can be seen that the value function is a solution of 
inf {G a V(t, x) + £(t, x, a)} = 0, (t, x) G Q, 

V(t, x) = g(t, x), (t, x) G d*Q = [0, T) x dO U {T} x O, 
in the sense, which we will now describe. 

Definition 2.1. Let u(t,x) = g(t,x), (t,x) G d*Q. (i) It is called a viscosity subsolution of (|2.3|) 
if for any (to, xq\ ip) G Qx C 2,l (Q) such that (p(t, x) < u(t, x), (t, x) G Q, and ip(to, xq) = u(to, xq) 
we have that 

inf {G>(*0, %o) + £(to, Xq, a)} > 0. 

(ii) It is called a viscosity supersolution of (|2.3|) if for any (to,xo;(p) G Q x C 2,1 (Q) such that 
(f(t,x) > u(t,x), (t,x) G Q, and tp(to,xo) = u(to,xo) we have that 

inf {G a <p{t , x ) + £(t , xq, a)} < 0. 

(Hi) Finally, u is a viscosity solution if it is both a viscosity subsolution and a viscosity superso- 
lution. 

Proposition 2.2. Suppose V(t, x) G C(Q) and Assumption 12.11 hold. Then the value function 
V(t, x) is the unique viscosity solution of (I2.3p . 

A complete proof of Proposition 12. 21 can be found in |4J. In Appendix, we provide an alternative 
proof for the existence part. 

The characterization of the value function in Proposition 12.21 assumes that it is continuous. 
However the value function is not necessarily continuous if the domain is a bounded set (see 
Figure [T] and Example 14. 1 j) . In the next section we give a sufficient condition that guarantees the 
continuity of the value function. This improves on the condition provided in Section V.2 of [1]. 

3. Sample Path Behavior on the boundary of domain 

In this section, we will discuss the sample path behavior of Ito process on [0, T) x dO, which 
turns out to be crucial for the continuity of the value function. 

For a given constant vector a G A, let Y be the unique strong solution of the following stochastic 
differential equation: 

dY s = b(s,Y s ,a)ds + a(s, Y s , a)dW s , Y t = y. 
The main result of this section, which we will state next, derives a sufficient condition (j3.2|) . 
under which the process Y must hit O c infinitely many times in any small duration, if it starts 
on dO. To formulate our result, let us denote the signed distance function by 



A f<kst(y,0), y O; 
^-dist(y,O c ), y G O. 
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Proposition 3.1. Let (t,y) € [0, T) x dO and a e A. Assume that dO £ C 2 and i/iai 

max{L?p(y), y, a)L>p(y)||} > 0. (3.2) 

Then, 

inf{s > t : Y s (£ 0} = t P - a.s. (3.3) 

Remark 3.1. The assumption that dO € C 2 implies that p £ C 2 in a neighborhood of dO; see 
Lemma 14-16 in [5]. Also see page 78 of j9] and the references therein. 

Before we present the proof of this proposition, we will need some preparation. First, note that 
(|3.3p can be written as the local behavior of a one-dimensional process p(X s ): 

inf{s > t : p(Y s ) > 0} = t P - a.s. 

Next, we will focus on one-dimensional process, which implies that a non-degenerate continuous 
local martingale process M starting from zero hits (0, oo) infinitely many times in any small time 
period. If M is a standard Brownian motion, the proof is given by Blumenthal 0-1 law (3j Theorem 
7.2.6]. However, because the distribution of M is not explicitly available, we use the representation 
of M as a time changed Brownian motion. 

Lemma 3.1. Let B(r) be a one- dimensional Brownian motion with respect to (Q, J-, F,Q). We 
assume that a is a one- dimensional progressively measurable process with J. a 2 dr < oo, so that 
M s = J. s & r dB r is a local martingale. Furthermore, we assume that a s > Vs € [t,T] Q-a.s. 
Then r = inf{s > t : M s > 0} satisfies r = t Q-a.s. 

Proof. First, we can extend function a on [t, T] to [t, oo) by a(s) = cr{T) for all s > T. Then, the 
quadratic variation of M is a strictly increasing function and it satisfies 

(M) s = J a 2 (r)dr — > oo as s — > oo, Q — a.s. 

since a > 0. For a given positive s, define T(s) = inf{r > : (M)(r) > s}. The strictly increasing 
function T satisfies T((M)(s)) = s. The time-changed process B s = Mx( s ) is a Q-Brownian 
motion under the filtration Q s = J^xts) an d M s = B/ M w s y, see e.g. [7, Theorem 3.4.6]. Thus, 
Q-almost surely, we have 

inf{s : M s > 0} = inf{s : B {M)(s) > 0} 

= mf{T(<M)( S )):£ (M)(s) >0} 
= T (inf{(M)( S ) : B {M){s) > 0}) 
= T(0) = 0. 

The second equality follows from the fact that a > 0. The third, on the other hand, follows from 
the fact that T is increasing. □ 

We are ready to prove Proposition 13.11 

Proof of Proposition 13.11 We will carry out the proof in two steps. 

(i) Let us first assume that \\a'(t,y,a)Dp(y)\\ > 0. Due to the continuity of this function, there 
exists a stopping time r > t, (which is less than the exit time from the neighborhood mentioned 
in Remark 13. ip such that for s £ (t, t) 

\\a'(s, Y s , a)Dp(Y s )\\ >e± ~\\o-'(t,y,a)Dp(y)\\ > 0, P - a.s. (3.4) 



Thus, applying Ito's formula, we obtain 

p(Y s ) = I L a r p{Y{r))dr+ I Dp(Y(r))a(r,Y(r),a)dW(r) 



L a rP (Y(r))dr+ / \\a'(r,Y(r), a)Dp(Y(r))\\dW(r) 
t Jt 

where TV is a one-dimensional P-Brownian motion. By Girsanov's theorem, there exists Q ~ P, 
such that 

p(Y s ) = J \\a'(r,Y(r),a)Dp(Y(r))\\dW r Q 

where Wfr is a Q-Brownian motion. Thus, p(Y s ) is a local martingale process under Q. Lemma [3. II 
implies that 

inf{s > t : p(Y s ) > 0} = t, Q - a.s. 
Since P is equivalent to Q, and the conclusion holds P-a.s. 

(ii) This was a case already proved in [3J Lemma V.2.1]. □ 

4. Continuity of the value function 
We will construct a sequence of functions that converge uniformly to the value function. For 
this purpose let d(x) = p + {x) and define A £ (s,X) = exp / — J d(X r )dr\ . Let 

J £ (t,x,a) =E ttX y\ £ (s,X)e( S ,X s ,a s )ds + A £ (T,X)g(T,X(T))} . (4.1) 

and 

V £ (t,x) =mf aeA J e (t,x,a). (4.2) 
Next, Lemma |4. II shows the continuity of this function. Its proof is given in the Appendix. 
Lemma 4.1. Under Assumption \2.1\ V s G C([0,T] xO). In fact, 

IV^^x 1 ) - V%t 2 ,x 2 )\ < C £ {\\x l -x 2 \\ + \h - h^ 2 ), 
for some positive constant C £ . 

Theorem 4.1. Assume that Assumption ^. II and the following hold: 

(1) dO G C 2 ; 

(2) V(i,x) G [0,T) x dO, there exists an a £ A satisfying ()3.2[) .- 
(3) 

inf {G a (u + g)(t, x) + £(t, x,a)}> 0, V(t, x) G [0, T] x M n . (4.3) 
Then V is continuous on Q. 

Remark 4.1. In [4, Pages 202-203], a sufficient condition for the continuity of the value function 
is given: Lfp(y) > for some a G A for all (t,y) G [0, T) x dO. Theorem 14.11 provides an 
alternative sufficient condition: \\a'(t,x,a)Dp(x)\\ > for some a G A for all (t,x) G [0,T) x dO. 
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Proof. The proof is divided into two steps. 

(i) Assume that I > 0, g = on R + x 1" x A. Fix (t,x) G [0,T) x dO. Let a £ A satisfy (pT2)) . 
Consider the constant control process {a s = a : s > t} and let Y denote the corresponding process 
governed by this constant control. By Theorem 13.11 for s G (t,T] we have 



J d(Y r )dr > 0, P - a.s. 



Hence, 

lim £ ^ + A £ (s,Y) = P - a.s. 
By Dominated Convergence Theorem, one can conclude that 

lim e _ >0 + Et,* { A £ (s, y s , a)ds + A £ (T, F) 5 (T, Y T )} = 0. 

This implies 

lim J £ (t, x, a) = 0. 

e-»0+ 

Together with J £ (t,x,a) > F £ (t,x) > which follows from (|4.2j) . the above implies that 

lim V £ (t, x) = = V(t, x), (t, x) G [0, T] x dO. (4.4) 

Therefore, V s (t, x) is continuous (Lemma 14. XH on the compact set [0, T] x dO in M n+1 , and it 
monotonically converges to the zero function. Dini's theorem implies that lim e ^ + V £ {t,x) = 
uniformly on [0, T] x dO. Thanks to the uniform convergence, if we set 

h{e) = sup{T/ £ (t,x) : (t,x) G [0,T] x dO}, 

we have that lim e ^ + h( £ ) = 0- 

Now we are ready to prove the continuity of the value function V. Let (t, x) G Q. Applying 
the dynamic programming principle to V £ (-, ■) with respect to stopping time r of ([1.2)1 . and using 
the fact that A e (s, X a ' x ' , ct s ) = 1 for s < r and a G A, we obtain 



F £ (t,x) = M\E t>x 



< inf < E 

aeA 



^ £(s, X^' a , a s )ds + V £ (t, X^ a ) | 

+ h(e), since {T,X t T ' x ' a )£d*Q 



(4.5) 



£(s,Xl' x ' a ,a s )ds 

= V(t,x) + h(e). 

Since I > 0, we further have that 

V(t, x) < V £ (t, x) < V(t, x) + h(e), V(i, x) G Q 

This implies y e — > V uniformly on Q. Since V s is continuous by Lemma 14.1) the value function 
V is also continuous. 



(ii) The proof follows from (i) once we let l(t,x,a) = l(t,x,a) + G a g(t,x) and consider (jl.3p 
and (|4.2|) by setting I = I and g = 0. 

□ 

Next, we give an example, whose value function is continuous, although it does not satisfy the 
sufficient condition of [1]. In this example, we first consider a deterministic exit time problem. 
We observe that this problem does not have a continuous value function. Next, we consider 
a degenerate random version of the same problem. In this problem, the sufficient condition 
Lfp(x) > of [1] holds only for some points x on the boundary. Yet, it still satisfies the sufficient 
condition of (|3,2p on the entire boundary, and therefore, the value function is continuous. 



Example 4.1. (i) Let Xt' x , s >t, be the one-dimensional process satisfying 

dx t,x = _ 2 ( s _ x t,x = x _ 

Let Q = [0, 2) x (-1, 1), and t 1 > x = inf{s > t : X^ x (s) g (— 1, 1)}. Let us define the value function 
as V(t, x) = {T t,x A 2) — t. Then, X t,x has an explicit form: 

Xl> x = -( s -l) 2 + x+ (t-lf. 

Therefore, the function s — > X s ' x first increases towards its maximum 

maxX*' 1 ' = x + (t - l) 2 , 

s>t 

and upon reaching it decreases to — oo. Thus, if x + (t — l) 2 > 1, then X = 1, otherwise 
IX = -1. As a result, for t G [0, 1], V(t, x) is discontinuous at every point on the parabola 

Ut,x) e Q : maxl s = l| = { (i, x) G Q : x = -t 2 + 2t} . 



We also note that, (|3.2p does not hold, since 

max{L^(±l), \\a'(t,x,a)p(±l)\\} = 0, Vt G (0,1). 



(ii) Next, we consider the following state process, which we obtain by adding a random pertur- 
bation to the above deterministic process: 

dX t,x = _ 2 ( s _ ^ ds + ( 2s _ x^+dWs, X\' x = x. 

This equation admits a unique strong solution since the coefficients are Lipschitz continuous. Let 
us define the value function to be V(t,x) = K tjX [{T t,x A 2) — £]. Note that, p(-) of (|3.ip satisfies 

p(x) = {x- l)l {x > } + (-1 - x)l {x<0} , Dp(x) = sgn{x), and D 2 p(x) = 0. (4.6) 

As a result, 

L t p{\) = -2(t - 1) > on t G (0, 1); \a(t, l)Dp{\)\ = (2t - 1) + > on t G (1/2, 2), 

and 

L t p(-1) = 2{t - 1) > on t G (1, 2); |<r(t, l)Dp{-\)\ = (2t + 1)+ > on t G (0, 2). 

Although, the condition Lfp > 0, which is the sufficient condition given by [3] — see equation 
(2.8) on page 202 — fails on the boundary, the continuity of the value function follows from 
Theorem 14.11 □ 

5. Appendix 

5.1. Proof of Proposition [2T2l First, we will develop the following auxiliary result. 

Lemma 5.1. For a given (t,x) G Q, define 

9 = inf{s > t : (s, X s ) <£[t,t + h 2 ) x B(x, h)}, 

where B(x, h) is a ball centered at x with radius h G (0, 1). Then, there exists a constant K, which 
does not depend on the control a, such that 

Et >a .[0-t] > Kti 2 . 



Proof. Let f(y) = \\y — x\\ 2 . Applying Ito's formula and taking expectations yield 

Et,*{/(*«>) " /(*)} = E tyX U L?f(X s )ds\ . (5.1) 

Since [t,t + 1] x B(x, 1) x A is compact, by continuity 

sup \Lsf( x )\ < K t,x < oo, 

(w)e[t,t+l]xB(i,l)xA 

for some constant lft )X . Since (s,X s ,a s ) £ [t,t+ 1] x -B(x, 1) x ^4 for any s G [i, 9] the integrand 
in (|5,1|) is bounded above by Kt yX . Since f(x) = 0, we can write (|5.ip as 



< K t:X E ttX [6-t]. 



E tjX [l {9=t+h2} f(X e )] + E t>x [l {e<t+h 2 } h 2 ] = E tyX J L a °f(X s )ds 
On the other hand, 

E t>x {9 -t)> E t>x [(9 - t)l {e=t+h2} ] = h 2 Et )X [l {e=t+h 2 } ] . 
Adding the last two inequalities, we get 

(K tiX + l)E t , x [9 -t}> h 2 +E t , x [l {e=t+h 2 } f(X e )] > h 2 . 

The result follows by setting K = l/(K t)X + 1)- □ 

Now, we are ready to prove Proposition 12.21 
Proof of Proposition 12.21 

(i) We will first show that V is a subsolution of (|2.3p . We will prove the assertion by a contra- 
diction argument. Let us assume that there (t,x;<p) as in Definition 12. 11 - (i) such that 

£(t,x,a) + G a <p(t,x) < -5, 

for some 5 > 0. Then, by continuity of I + G a ip in (t, x), there exists h > such that 

£(s, y, a) + G a ip{y, a) < -- < 0, V(s, y) e[t,t + h 2 ) x B(x, h) C Q. 

Let Y be the process which can be obtained by applying the control a = a and define 

9 = inf{s > t, Y s £ B(x, h)} A (t + /i 2 ). 
By the dynamic programing principle 

^(t, x) < Et,* |^ ^(s, n, a)ds + V{9, Y e ) 
It follows from how (p is chosen that 

< e m{^ %y s ,a)d S + ^(0,y e )-^,x) 

which yields a contradiction. 

(ii) We will now show that V is a supersolution of f|2 . 3j) . We will, again, use proof by con- 
tradiction. Let us assume that there exists a triplet (t,x;<p) as in Definition 12. H - (ii) such that 

inf {lit, x, a) + G a tp(t, x)} = 5 > 0, 



- ) ds } < 0, 



As a function of (t, x), £(t, x, a) + G a ip{t, x) is equicontinuous in A, by Assumption l2.1i Therefore, 

M{£(t,x,a) + G a ip(t,x)} 

aeA 

is also continuous in (t, x). So, one can find h > such that 

inf {£(s, y, a) + G>(s, y)} > J > 0, V(s, y) e[t,t + h 2 ) x £(x, fc). 

aeA 2 

Let e = jKh 2 , where K is the constant in Lemma 15.11 Let a be e-optimal control and define 



6 = inf{s > t : X s <£ B(x, h)} A (t + /i 2 ). 



Then 



y(t,x) >E t , 



In the following, we obtain the desired contradiction: 



J £{s,X s ,a s )ds + V(9,X e )\-e, 



> e m{^" £(s,X at a a )ds + ip(0,X )-<p(t t x)\-e, 
= E ^W Ws,X s ,a s ) + G^( S ,X s )]<feJ -e 

>E*^{j[ Ws,^ s ,a s ) + G a V(s,^)]<fej byLemmaO 



^,X s ,a s ) + G a ^(s,X s ) 



ds 



> -Et tX [e-t] >o. 



□ 



5.2. Proof of Lemma 14.11 First, it can be checked that the following inequality holds: 

^(x 1 ) - d{x 2 )\ < Wx 1 - x 2 \\, x x ,x 2 G R n . 

As a result 



lA^A^-A^A 2 )! 



cxp 

\-\J d{Xl)dr\ -exp|-i J d(X 2 )d; 

- [ \\X} -X 2 \\dr 
£ it 



d(X}) - d(X 2 )dr 



1 

< - 

e 

<-(s-t) sup \\X}-X 2 \\. 

e re[t,s] 
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For (p = i, g we have that 

E t>x { \A £ (s, X l )tp(s, Xl) - A £ (s, X 2 Ms, X 2 ) | } 

<E i ,4|(A e ( S ,X 1 )-A £ ( S ,X 2 ))^( S ,Xi)|}+E^{|A e ( S ,X 2 )(^( S ,Xi)-^,X s 2 ))|} 
< (E^IA^^X^-A^s,^ 2 )! 2 ) 172 ^ x \^X])\ 2 ) l,2 + 

(E t , x \A £ (s,X 2 )\ 2 ) 1/2 (E^( S ,Xj)-<^,X 2 )| 2 ) 1/2 



1 



1/2 



1 vl|2\l/2 



< -(s - t) E tjX { sup \\X^ - X 2 \\) 2 + K (E tjX \Xl - X l s \ 2 ) 

< Clx 1 -x 2 \, 

for some positive constant C. In the above derivation, we utilized 



E[sup t < s < tl \\Xl-X 2 \\ 2 ] <C\\x l 



,.2112 



t<h<T, 



for another positive constant C. Now, we are ready to prove the regularity of V s in x. For any 

x x ,x 2 £ O, 



\V E (t,x l )-V E (t,x 2 )\ <sup{E ttX 



\A £ {s, X 1 )^, X l {s),a s ) - A £ (s, X 2 )£{s, X 2 {s),a s )\ds 



+ E t>x [^(^X^g^X^T^-A'^X^g^X 2 ^] j 
KCWx 1 -x 2 \\, 

for some positive constant C. Please refer to [10] for the moment inequalities we used above. 

Let us prove the regularity of the value function in t. For ti < t 2 , we can use the dynamic 
programming principle to write 

\V £ (h,x)-V £ (t 2 ,x)\ <sup [ * E t , x \A £ (s,X)t(s,X s ,a s )\ds + supE t , x \V £ (t 2 ,X(t 2 )) -V £ {t 2 ,x)\ 



< C 



rt2 

sup / E t , x (1 + ||X,|| 2 ) ds + E tyX \\X t2 - x\ 

. " Jti 



< C x {t 2 - h) + C 2 {t 2 - h) 1 ' 2 < (CiT + C 2 )(t 2 - h) 1 / 2 , 
in which C, C\ and C 2 are positive constants. Here, we used the facts that 



E 



sup \\X S 

0<s<T 



< CO, 



and 



for some constant C. 



supE^UI^-sU] < C[s - il 1 / 2 , 



□ 
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